Smallest Reduction Matrix of Binary Quadratic Forms And Cryptographic Applications

We present a variant of the Lagrange-Gauss reduction of quadratic forms designed to minimize the norm of the reduction matrix within a quadratic complexity. The matrix computed by our algorithm on the input f has norm O f 1 2 Δ 4 f , which is the square root of the best previously known bounds using classical algorithms. This new bound allows us to fully prove the heuristic lattice based attack against NICE Cryptosystems, which consists in factoring a particular subclass of integers of the form pq. In the process, we set up a homogeneous variant of Boneh-Durfee-HowgraveGraham’s algorithm which finds small rational roots of a polynomial modulo unknown divisors. Such algorithm can also be used to speed-up factorization of pq for large r.